There is a conventional technology in which pickup images obtained by a camera or suchlike are used in a game. For example, a game apparatus provided with a camera can use a pickup image obtained by a camera as a portion of a game image. In the game apparatus, for example, the camera picks up an image of the game player's face, and then uses the pickup image in game processing as a game input, or displays the pickup image after the game. Accordingly, the player can use the pickup image in a game operation or, following the game, enjoy viewing the expression on the face captured during the game.
While conventionally, pickup images are used as game images, the real space as represented by the pickup images is not displayed in good association with a virtual space constructed by the game apparatus. For example, while conventionally, pickup images are displayed with objects or suchlike synthesized thereon, it is not possible to allow the player to have a feeling as if the virtual space displayed on the screen actually existed in depth beyond the screen, and therefore, in some cases, the virtual space might not appear sufficiently real.
Therefore, the game systems, game apparatuses, storage media having game programs stored therein, and image generation methods as disclosed herein are intended to allow a virtual space displayed on the screen to appear more real.
(1) An example game system described herein includes a game apparatus and a terminal device capable of communicating with the game apparatus.
The terminal device includes an image pickup unit, a transmission unit, and a display unit. The transmission unit transmits a pickup image obtained by the image pickup unit to the game apparatus. The display unit displays an image outputted by the game apparatus.
The game apparatus includes a reception unit, a first image generation unit, a first image output unit, a second image generation unit, and a second image output unit. The reception unit receives the pickup image. The first image generation unit generates a first image by synthesizing the pickup image at least in part with an image of a predetermined virtual space as viewed from a first virtual camera. The first image output unit outputs the first image to a predetermined display device. The second image generation unit generates a second image of the virtual space as viewed from a second virtual camera different from the first virtual camera. The second image output unit outputs the second image to the terminal device.
The “game apparatus” may be any apparatus capable of performing the processing of each of the aforementioned units. For example, the game apparatus may be an information processing apparatus exclusively intended for game use or may be a multipurpose information processing apparatus such as a general personal computer.
The “terminal device” is a device including an image pickup unit and a display unit and being capable of communicating with the game apparatus, and may have the function of an operating device as does a terminal device 7 in an example embodiment to be described later.
The “game system” includes an operating device and an information processing apparatus, and may or may not include the “predetermined display device” for displaying the first image. That is, the game system may or may not be provided in a form including the predetermined display device.
The “first image” may be generated arbitrarily so long as it includes an image representing a virtual space and at least a part of a pickup image. The “first image” may be perceived as if the real space represented by the pickup image were connected to the virtual space, e.g., the first image has the pickup image (at least in part) arranged in the virtual space. Here, the “image having the pickup image arranged in the virtual space” may be generated arbitrarily so long as the pickup image appears to exist in the virtual space. The “first image” may or may not have a virtual space object displayed on (or in front of) the pickup image.
The “second image” represents the virtual space as viewed from the second virtual camera different from the first virtual camera. That is, the “second image” represents the virtual space viewed from a different viewpoint from the first image.
According to the above configuration (1), the first image to be displayed on the predetermined display device is an image obtained by synthesizing an image of the virtual space and a pickup image, and therefore, the first image represents the real space as viewed from the virtual space. On the other hand, the second image to be displayed on the terminal device is an image of the virtual space viewed from a different viewpoint from the first image, and therefore, the second image represents the virtual space as viewed from the real space. As a result, the predetermined display device displays an image of the real space as viewed from the virtual space, and the terminal device displays an image of the virtual space as viewed from the real space. Thus, by viewing these two images, the player holding the terminal device can feel as if the virtual space were connected to the real space and actually existed. The player can clearly feel as if the virtual space spread beyond the screen of the terminal device, and can perceive the virtual space to be more real. Thus, according to the above configuration (1), the terminal device displays a virtual space, and the predetermined display device displays an image obtained by synthesizing a pickup image with an image representing the virtual space, so the virtual space can appear more real.
(2) The game apparatus may further include a virtual camera setting unit. The virtual camera setting unit sets the first virtual camera and the second virtual camera such that the first and second virtual cameras are directed from opposite sides toward a predetermined plane within the virtual space.
The “predetermined plane” is a concept used for specifying the positional relationship between two virtual cameras, and the actual game processing does not include arranging an object corresponding to the predetermined plane or calculating the position and/or the attitude of the predetermined plane. That is, the virtual camera setting unit sets two virtual cameras, and may or may not perform the processing for arranging the predetermined plane.
According to the above configuration (2), the second image is an image of the virtual space representing the predetermined plane as viewed from one side, and the first image is an image of the virtual space representing the predetermined plane as viewed from the other side and including a pickup image within the virtual space. In this case, the line-of-sight direction in the second image is from the front side of the screen of the terminal device to the rear, and is the same as the player's line-of-sight direction. On the other hand, with reference to the virtual space displayed on the terminal device, the line-of-sight direction in the first image is from the rear side of the terminal device's screen toward the front, and is approximately opposite to the player's line-of-sight direction. That is, the first image is provided as an image as viewed from the rear side of the terminal device's screen toward the player. Thus, according to the above configuration (2), the terminal device displays an image with the line-of-sight direction from the front of the terminal device's screen toward the rear, and the predetermined display device displays an image with the line-of-sight direction from the rear of the screen toward the front, so the virtual space represented by each image can appear more real.
(3) The first image generation unit may generate a first image with the pickup image being displayed at least in part at a position of the predetermined plane.
According to the above configuration (3), the pickup image is displayed at the position of the predetermined plane, and therefore, the player has a perception as if the virtual space existed on the side of the first virtual camera with respect to the predetermined plane, and as if the real space were on the side of the second virtual camera. Thus, the player can feel as if the virtual space were connected to the real space via the predetermined plane, so the virtual space can appear more real.
(4) The game apparatus may further include a plane arrangement unit for arranging a rendering plane within the virtual space, the plane having at least apart of the pickup image rendered thereon. In this case, the first image generation unit generates a first image which represents the virtual space including the rendering plane.
The “rendering plane” is an arbitrary plane within the virtual space, and may have any arbitrary shape, e.g., a curved shape, or a rectangular plane as in the example embodiment to be described later. Moreover, the pickup image may be rendered on the whole or part of the rendering plane.
The above configuration (4) makes it possible to readily generate a first image which can appear as if (at least a part of) the pickup image existed in the virtual space.
(5) The game apparatus may further include a first camera setting unit for setting a position and an attitude of the first virtual camera such that the rendering plane is viewed from a side on which the pickup image is rendered.
According to the above configuration (5), the first virtual camera has the virtual space including the rendering plane placed within its field-of-view range, making it possible to readily generate a first image by synthesizing an image of the virtual space with a pickup image. Moreover, the first image can be viewed as if the virtual space were connected to the real space represented by the pickup image, so the virtual space can appear more real.
(6) The game apparatus may further include a second camera setting unit for setting a position and an attitude of the second virtual camera such that the rendering plane is viewed from an opposite side to a side on which the pickup image is rendered.
The above configuration (6) makes it possible to readily generate a second image of the virtual space not including a pickup image. Particularly, in the case where the above configuration (6) is used in combination with the above configuration (5), the first and second virtual cameras are set so as to face approximately opposite directions with respect to the rendering plane in the virtual space. Thus, as in the above configuration (2), the terminal device displays an image with the line-of-sight direction from the front of the screen toward the rear, and the predetermined display device displays an image with the line-of-sight direction from the rear of the screen toward the front, so the virtual space represented by each image can appear more real.
(7) The game system may further include a game processing unit for executing predetermined game processing on the basis of an action of a predetermined object arranged in the virtual space and the pickup image rendered on the rendering plane.
According to the above configuration (7), when the predetermined object takes some action toward the pickup image, the player can perceive the object in the virtual space affecting the real space represented by the pickup image. Thus, the player can strongly feel as if the virtual space were connected to the real space, so the virtual space can appear more real.
(8) The game system may further include an operating device capable of communicating with the game apparatus. In this case, the game processing unit controls the action of the predetermined object in accordance with an operation on the operating device.
The above configuration (8) allows a player different from the player holding the terminal device to play the game using the operating device. By manipulating the predetermined object using the operating device, the player can perceive the object in the virtual space affecting the real space represented by the pickup image.
(9) The game apparatus may further include a recognition processing unit and a game control processing unit. The recognition processing unit performs image recognition processing on the pickup image. The game control processing unit performs game control processing to add a change to the virtual space on the basis of the result for the image recognition processing.
According to the above configuration (9), the virtual space is changed on the basis of the result for the image recognition processing performed on the pickup image. Accordingly, the player can perform a game operation by changing the content of a pickup image to be obtained by the image pickup unit. For example, the player can perform a game operation by changing the expression on the face or by changing the position of the face. Here, the pickup image is displayed on the predetermined display device, along with an image of the virtual space, and therefore, the player's game operation is reflected in the virtual space displayed on the predetermined display device. That is, the above configuration (9) allows the player's action in the real space to be reflected in the displayed image, thereby changing the virtual space in accordance with the player's action, so the virtual space can appear more real.
(10) The terminal device may further include a microphone. In this case, the transmission unit transmits data for sound detected by the microphone to the game apparatus. Moreover, the game apparatus further includes a game control processing unit for performing game control processing to add a change to the virtual space on the basis of the sound data.
According to the above configuration (10), the virtual space is changed on the basis of sound detected by the microphone. Thus, the player can perform a game operation by providing an input to the microphone.
(11) The transmission unit may transmit operation data representing a player's operation on the terminal device to the game apparatus. In this case, the game apparatus further includes a game control processing unit for performing game control processing to add a change to the virtual space on the basis of the operation data.
The above configuration (11) allows the player to readily perform a game operation while holding the terminal device.
(12) The terminal device may further include a sensor unit for outputting motion data representing information about motion of the terminal device. In this case, the transmission unit transmits operation data including the motion data to the game apparatus.
The above configuration (12) allows the player to perform a game operation by moving the terminal device, making it possible to play the game with intuitive and easy operation.
(13) The terminal device may further include a position detection unit capable of detecting an input position on a screen of the display unit. In this case, the transmission unit transmits operation data including data for the position detected by the position detection unit to the game apparatus.
The above configuration (13) allows a game operation to be performed by directly touching the virtual space displayed on the screen of the terminal device, making it possible to play the game with intuitive and easy operation.
(14) The terminal device may further include at least one operation member. In this case, the transmission unit transmits operation data representing the player's operation on the operation member to the game apparatus.
The above configuration (14) allows the player to readily perform a game operation while holding the terminal device.
Note that disclosed herein are examples of the game apparatus included in the game system and the storage medium having stored therein a game program for causing a computer to function as means equivalent to the aforementioned units of the game apparatus (excluding the reception unit and the image output units). Also disclosed herein is an example of the image generation method to be executed in the game system or apparatus. Further disclosed herein are examples of the information processing system, apparatus, and program intended for purposes including, but not limited to, game use, and the image generation method to be executed by the information processing system or apparatus.
In the game system, game apparatus, game program, and image generation method as mentioned above, the predetermined display device displays a first image obtained by synthesizing an image of the virtual space as viewed from the first virtual camera with at least a part of a pickup image obtained by the terminal device, and the terminal device displays a second image of the virtual space as viewed from the second virtual camera different from the first virtual camera. Thus, the player can feel as if the virtual space were connected to the real space represented by the pickup image, so the virtual space can appear more real.
These and other objects, features, aspects and advantages will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings.